Conservative Safety Monitors of Stochastic Dynamical Systems

Generating accurate runtime safety estimates for autonomous systems is vital to ensuring their continued proliferation. However, exhaustive reasoning about future behaviors is generally too complex to do at runtime. To provide scalable and formal safety estimates, we propose a method for leveraging design-time model checking results at runtime. Specifically, we model the system as a probabilistic automaton (PA) and compute bounded-time reachability probabilities over the states of the PA at design time. At runtime, we combine distributions of state estimates with the model checking results to produce a bounded time safety estimate. We argue that our approach produces well-calibrated safety probabilities, assuming the estimated state distributions are well-calibrated. We evaluate our approach on simulated water tanks

Curating Naturally Adversarial Datasets for Trustworthy AI in Healthcare

Deep learning models have shown promising predictive accuracy for time-series healthcare applications. However, ensuring the robustness of these models is vital for building trustworthy AI systems. Existing research predominantly focuses on robustness to synthetic adversarial examples, crafted by adding imperceptible perturbations to clean input data. However, these synthetic adversarial examples do not accurately reflect the most challenging real-world scenarios, especially in the context of healthcare data. Consequently, robustness to synthetic adversarial examples may not necessarily translate to robustness against naturally occurring adversarial examples, which is highly desirable for trustworthy AI. We propose a method to curate datasets comprised of natural adversarial examples to evaluate model robustness. The method relies on probabilistic labels obtained from automated weakly-supervised labeling that combines noisy and cheap-to-obtain labeling heuristics. Based on these labels, our method adversarially orders the input data and uses this ordering to construct a sequence of increasingly adversarial datasets. Our evaluation on six medical case studies and three non-medical case studies demonstrates the efficacy and statistical validity of our approach to generating naturally adversarial dataset

Compositional Probabilistic Analysis of Temporal Properties over Stochastic Detectors

Run-time monitoring is a vital part of safety-critical systems. However, early-stage assurance of monitoring quality is currently limited: it relies either on complex models that might be inaccurate in unknown ways, or on data that would only be available once the system has been built. To address this issue, we propose a compositional framework for modeling and analysis of noisy monitoring systems. Our novel 3-value detector model uses probability spaces to represent atomic (non-composite) detectors, and it composes them into a temporal logic-based monitor. The error rates of these monitors are estimated by our analysis engine, which combines symbolic probability algebra, independence inference, and estimation from labeled detection data. Our evaluation on an autonomous underwater vehicle found that our framework produces accurate estimates of error rates while using only detector traces, without any monitor traces. Furthermore, when data is scarce, our approach shows higher accuracy than non-compositional data-driven estimates from monitor traces. Thus, this work enables accurate evaluation of logical monitors in early design stages before deploying them

Causal Repair of Learning-enabled Cyber-physical Systems

Models of actual causality leverage domain knowledge to generate convincing diagnoses of events that caused an outcome. It is promising to apply these models to diagnose and repair run-time property violations in cyber-physical systems (CPS) with learning-enabled components (LEC). However, given the high diversity and complexity of LECs, it is challenging to encode domain knowledge (e.g., the CPS dynamics) in a scalable actual causality model that could generate useful repair suggestions. In this paper, we focus causal diagnosis on the input/output behaviors of LECs. Specifically, we aim to identify which subset of I/O behaviors of the LEC is an actual cause for a property violation. An important by-product is a counterfactual version of the LEC that repairs the run-time property by fixing the identified problematic behaviors. Based on this insights, we design a two-step diagnostic pipeline: (1) construct and Halpern-Pearl causality model that reflects the dependency of property outcome on the component's I/O behaviors, and (2) perform a search for an actual cause and corresponding repair on the model. We prove that our pipeline has the following guarantee: if an actual cause is found, the system is guaranteed to be repaired; otherwise, we have high probabilistic confidence that the LEC under analysis did not cause the property violation. We demonstrate that our approach successfully repairs learned controllers on a standard OpenAI Gym benchmark

Distributionally Robust Statistical Verification with Imprecise Neural Networks

A particularly challenging problem in AI safety is providing guarantees on the behavior of high-dimensional autonomous systems. Verification approaches centered around reachability analysis fail to scale, and purely statistical approaches are constrained by the distributional assumptions about the sampling process. Instead, we pose a distributionally robust version of the statistical verification problem for black-box systems, where our performance guarantees hold over a large family of distributions. This paper proposes a novel approach based on a combination of active learning, uncertainty quantification, and neural network verification. A central piece of our approach is an ensemble technique called Imprecise Neural Networks, which provides the uncertainty to guide active learning. The active learning uses an exhaustive neural-network verification tool Sherlock to collect samples. An evaluation on multiple physical simulators in the openAI gym Mujoco environments with reinforcement-learned controllers demonstrates that our approach can provide useful and scalable guarantees for high-dimensional systems

Protective Cr Coatings with ZrO2/Cr Multilayers for Zirconium Fuel Claddings

This article described the protective properties of Cr coatings with a barrier layer composed of ZrO2/Cr multilayers deposited onto E110 zirconium alloy. The coatings with a ZrO2/Cr multilayer thickness of 100, 250, and 750 nm and single-layer (1.5 µm) ZrO2 barrier were obtained by multi-cathode magnetron sputtering in Ar + O2 atmosphere. Then, cracking resistance and oxidation behavior were studied under conditions of thermal cycling (1000 °C) in air and high-temperature oxidation at 1200-1400 °C in a water steam. The role of the ZrO2/Cr multilayers and multilayer thickness on cracking resistance of the experimental coatings and oxidation resistance of the coated E110 alloy was discussed. It was shown that the coatings with more quantity of the ZrO2/Cr multilayers have higher cracking resistance, but such types of samples have a large amount of coating spallation under thermal cycling. The high-temperature steam oxidation (1200-1400 °C) demonstrated that interfaces of the ZrO2/Cr multilayers can act as a source of cavities formed by the Kirkendall mechanism that results in accelerating Cr-Zr interdiffusion for Cr-coated E110 alloy

Single-window integrated development environment

International audienceThis paper addresses the problem of IDE interface complexity by introducing single-window graphical user interface. This approach lies in removing additional child windows from IDE, thus allowing a user to keep only text editor window open. We describe an abstract model of IDE GUI that is based on most popular modern integrated environments and has generalized user interface parts. Then this abstract model is reorganized into single windowed interface model: access to common IDE functions is provided from the code editing window while utility windows are removed without loss of IDE functionality. After that the implementation of single-window GUI on KDevelop 4 is described. And finally tool views and usability of several well- known IDEs are surveyed